Electronic communication system and method with sentiment analysis

ABSTRACT

Electronic communication methods and systems for determining sentiment associated with a communication, conveniently displaying indicia of the sentiment information, and scoring the sentiment information are disclosed. The methods and systems can include associating an emoji with a sentiment and annotating information, for example, a highlight reel or a waveform, with the emoji.

FIELD OF THE INVENTION

The present disclosure generally relates to electronic communicationmethods and systems. More particularly, examples of the disclosurerelate to electronic communication methods and systems suitable fordetermining sentiments associated with a communication and displayingindications thereof.

BACKGROUND OF THE DISCLOSURE

Call or contact centers can be used for a variety of applications,including taking orders, providing technical support, taking surveys,receiving complaints, and the like. Call center agents can communicatewith others, such as customers, using a variety of communicationtechniques, including voice calls, video calls, chats, texts, emails, orother electronic communication techniques.

During a call center communication, a sentiment of the customer and/orof the agent may vary during the course of the communication.Additionally or alternatively, sentiments or average sentiments ofagents, groups of agents, and/or customers can vary. Unfortunately, itcan be difficult to detect and/or determine sentiment trends forparticular agents, groups of agents, or for particular customers.Further, it can be difficult to determine how particular agents handlecommunications for certain sentiments, such as negative or aggressivesentiments. Accordingly, improved electronic communication systems andmethods that allow for relatively easy review and analysis of customerand/or agent sentiments are desired.

Any discussion of problems provided in this section has been included inthis disclosure solely for the purposes of providing a background forthe present invention and should not be taken as an admission that anyor all of the discussion was known at the time the invention was made.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The subject matter of the present disclosure is particularly pointed outand distinctly claimed in the concluding portion of the specification. Amore complete understanding of the present disclosure, however, may bestbe obtained by referring to the detailed description and claims whenconsidered in connection with the drawing figures, wherein like numeralsdenote like elements and wherein:

FIG. 1 illustrates an electronic communication system in accordance withexemplary embodiments of the disclosure.

FIG. 2 illustrates a portion of an electronic communication system inaccordance with exemplary embodiments of the disclosure in greaterdetail.

FIG. 3 illustrates a method in accordance with exemplary embodiments ofthe disclosure.

FIG. 4 illustrates another method in accordance with exemplaryembodiments of the disclosure.

FIG. 5 illustrates another method in accordance with further exemplaryembodiments of the disclosure.

FIG. 6 illustrates a running sentiment graph in accordance with furtherexemplary embodiments of the disclosure.

FIG. 7 illustrates an annotated waveform with one or more emojisindicative of sentiment in accordance with further exemplary embodimentsof the disclosure.

FIG. 8 illustrates examples of displayed information in accordance withfurther exemplary embodiments of the disclosure.

It will be appreciated that elements in the figures are illustrated forsimplicity and clarity and have not necessarily been drawn to scale. Forexample, the dimensions of some of the elements in the figures may beexaggerated relative to other elements to help to improve understandingof illustrated embodiments of the present disclosure.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The description of exemplary embodiments of the present disclosureprovided below is merely exemplary and is intended for purposes ofillustration only; the following description is not intended to limitthe scope of the invention disclosed herein. Moreover, recitation ofmultiple embodiments having stated features is not intended to excludeother embodiments having additional features or other embodimentsincorporating different combinations of the stated features.

As set forth in more detail below, exemplary embodiments of thedisclosure relate to electronic communication systems and methods. Theelectronic communication systems and methods can be used in a variety ofapplications, including use in call centers. Exemplary methods andsystems can determine one or more sentiments associated with anelectronic communication (e.g., various sections or portions of theelectronic communication) and display indicia of the sentiment(s) andcommunication information to a user, such as a call center agent or anagent supervisor. The indicia can be in the form of one or more emojis,which allows for universal (e.g., language-independent) review ofsentiments associated with a communication. The emojis can be presentedin the form of a highlight reel, which allows easy review of sentimentsand easy access to underlying (e.g., audio and/or transcribed)information associated with the communication. Further, methods areprovided for displaying information annotated with emojis. The emojiscan be linked to information, such as audio and/or transcribedinformation associated with a communication.

Turning now to the figures, FIG. 1 illustrates an electroniccommunication system 100 in accordance with exemplary embodiments of thedisclosure. For illustration purposes, system 100 is described in thecontext of a call center. However, unless otherwise noted, systems andmethods in accordance with embodiments of the disclosure are not solimited.

Electronic communication system 100 includes one or more devices, suchas devices 102, 108, and 110; a network 104; a communication server 106;and a sentiment determination system 112 coupled to communication server106. Electronic communication system 100 can include other devices,servers, networks, and the like, coupled to communication server 106and/or network 104.

Devices 102, 108, and 110 can be or include any suitable device withwired or wireless communication features and that can connect to network104 and/or communication server 106. For example, one or more of devices102, 108, 110 can be or include a wearable device, a tablet computer, awired phone, a mobile phone, a personal (e.g., laptop or desktop)computer, a streaming device, such as a game console or other mediastreaming device, or the like. One or more of devices 102, 108, 110 caninclude an application or client as to perform various functions setforth herein and/or to cause to be displayed text and/or otherinformation as described herein. By way of example, an application orclient can display information, annotated with one or more emojis, to auser in a manner as described in more detail below.

Network 104 can include or be, for example, an internet protocol (IP)network. Exemplary types of networks suitable for communication withnetwork 104 can be or include a local area network, a wide-area network,a metropolitan area network, one or more wireless networks, or a portionof the Internet. Various components of network 104 can be coupled to oneor more other components using an Ethernet connection, other wiredconnections, and/or wireless interfaces. Network 104 can be coupled toother networks and/or to other devices typically coupled to networks. Byway of particular example, network 104 includes a communication networkand network 104 can be coupled to additional networks that can becoupled to one or more devices, such as device 102. Exemplary additionalnetworks can include a network similar to network 104, a public switchedtelephone network (PSTN), or the like.

Communication server 106 can be or include any suitable server.Communication server 106 can perform various functions as describedherein. For example, communication server 106 can provide electroniccommunication (e.g., audio, video, text, and/or email) links betweendevice 102 and one or more devices 108, 110. In accordance with anexample of the disclosure, communication server 106 includes a privatebranch exchange (PBX) server. Further, communication server 106 iscommunicatively coupled to sentiment determination system 112 to receiveinformation from sentiment determination system 112 and include at leastsome of the information provided by communication server 106 to one ormore devices 108, 110.

In some cases, communication server 106 can include a switch+IVR module114 and optionally a user authentication module 116. Although separatelyillustrated, switch+IVR module 114 and user authentication module 116can be combined and/or various functions of one or more ofswitch+(optional) interactive voice response (IVR) module 114 and userauthentication module 116 can be separated into one or more additionalmodules and/or servers. In some cases, authentication module 116 can beused to authenticate an agent or a supervisor. Such authentication canbe used to allow access to various information and/or system functionsas described herein.

As used herein, “module” can refer to computer program instructions,encoded on computer storage medium for execution by, or to control theoperation of, data processing apparatus. Alternatively or additionally,the program instructions can be encoded on an artificially-generatedpropagated signal, e.g., a machine-generated electrical, optical, orelectromagnetic signal, which is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus. A computer storage medium can be, or be includedin, a computer-readable storage device, a computer-readable storagesubstrate, a random or serial access memory array or device, or acombination of one or more of the substrates and devices. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially-generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate physical components or media (e.g., solid-state memory thatforms part of a device, disks, or other storage devices).

Switch+IVR module 114 can be used to control call flows to and fromcommunication server 106. When used, an IVR component of module 114 canbe used to allow callers to initially interact with communication server106 via their telephone keypads and/or voice commands.

FIG. 2 illustrates a sentiment determination system 200 in accordancewith examples of the disclosure. Sentiment determination system 200 canbe the same or similar to sentiment determination system 112.

In the illustrated example, sentiment determination system 200 includesa call recorder 202, a transcription module 204, a sentiment extractionengine 206, a database 208, a highlight production module 210, anoverlay production module 212, a database 214, and a score (emoji score)calculation module 216. Sentiment determination systems in accordancewith other examples of the disclosure can include various combinationsof the components of sentiment determination system 200; such sentimentdetermination system 200 need not include all such components. Forexample, a sentiment determination system may not include an overlayproduction module and/or a highlight reel production module.

Call recorder 202 can be or include any suitable module for recording atleast a portion of a communication between two devices, such as device102 and one or more of devices 108, 110 to create audio informationassociated with a communication. In accordance with examples of thedisclosure, call recorder 202 can be automatically initiated once acommunication between two or more devices begins or once an agent device(e.g., one or more of devices 108, 110) is connected to a communication.In this case, sentiment determination system 200 can receive a signalfrom communication server 106 to indicate that a communication has beenestablished and to start recording. Similarly, sentiment determinationsystem 200 can receive a signal from communication server 106 toindicate that a communication has terminated and to stop recording.

Transcription module 204 can transcribe at least a portion of the audioinformation created by call recorder 202 to generate transcribedinformation. The transcription can be substantially real-time, such thatthe transcribed information is at least partially generated during acommunication. Alternatively, the transcribed information can begenerated at a later time, such as after the communication has ended, ondemand (e.g., when a report is requested), or the like.

Sentiment extraction engine 206 can analyze the transcribed informationand/or the audio information to determine sentiment information. Forexample, sentiment extraction engine 206 can review audio and/ortranscribed information and determine sentiment based on matchingpredefined words or phrases with words or terms stored in a database,such as database 208 or another database. The matching need not be withidentical terms. The match can be to synonyms stored within thedatabase. Additionally or alternatively, a loudness or a changes inloudness (e.g., 10 decibels or more or 15 decibels or more) can be usedto determine a sentiment, such as anger or frustration. In some cases,sentiment can perform machine learning and update words, phrases, and/oremojis associated with the words and phrases based on the learning.

Databases 208 and/or 214 can include one or more devices, such ascomputers or servers, to store information. By way of examples,databases 208, 214 can use one or more of Microsoft SQL Server, MySQL,Microsoft Access, Oracle, or the like relational database systems.Further, although separately illustrated, databases 208 and 214 can becombined and/or combined with other databases.

In accordance with examples of the disclosure, database 208 includes alist of sentiment terms and one or more emojis associated with thesentiment terms. For example, a negative or angry emoji

can be associated with terms, such as “not acceptable,” I want to “speakwith a manager,” “refund” or the like; a positive or happy emoji

can be associated with terms, such as “like,” “thank you,” and the like.Additionally or alternatively, emojis can be associated with an agent'sdialog. For example, a positive—e.g., high score emoji (e.g., a goldscript symbol) can be associated with a dialog that follows apredetermined script and/or a negative emoji (e.g., a black script) canbe associated with dialog that does not follow a script. The termsassociated with emojis can be in one or more languages. The table caninclude icons/emojis and/or associated words retrieved from othersources, such as the Unicode Consortium. Additionally or alternatively,the database can be generated by an enterprise. In these cases, thedatabase can include custom or unique emojis associated with words.Further, in some cases, an agent can add an emoji to a conversation(e.g., chat) with a customer. Such emoji's can be added to the databasewith associated words—e.g., 1-2 words preceding and/or 1-2 wordsfollowing the emoji. Table 1 below illustrates exemplary emojis andexemplary terms associated with the emojis. These examples are meant tobe illustrative, and, unless otherwise noted, are not meant to limit thescope of the invention. In addition, each emoji can have an associatedscore, which can be used to analyze sentiment information. The sentimentterms, emojis, and/or score information can be relationally stored indatabase 208.

TABLE 1 Term Emoji Score [−10 to 10] Angry

−10  Frustrated

−6 Annoyed

−3 Neutral

 0 Happy

 3 Elated

 6 Overjoyed

10

Highlight reel production module can access information from sentimentextraction engine and/or database 208 (e.g., emojis, audio information,and/or transcribed information) to form a highlight reel that includesone or more emojis associated with an electronic communication.

An exemplary highlight reel 600 is illustrated in FIG. 6. Highlight reel600 illustrates sentiment vs. time for an electronic communication. Inthe illustrated example, the vertical axis relates to sentiment and thehorizontal line corresponds to time. One or more emojis (e.g., emoji602) in highlight reel 600 can be linked to audio information and/ortranscribed information, which can be stored in database 208. An emoji604 corresponding to an average sentiment for a communication, can alsobe included in the highlight reel. Emoji 604 can be determined byaveraging (e.g., a weighted average) scores of the emojis withinhighlight reel 600. The averaging can be determined using a score, suchas the scores illustrated in Table 1. Highlight reel information can bestored in database 208 or database 214.

Overlay production module can be used to provide a waveform 700 of thecommunication with one or more emojis 702-706 corresponding to sentimentoverlaid on a waveform 708 to form annotated waveform 700, asillustrated in FIG. 7. A user—e.g., an agent or a supervisor—can selectan emoji 702-706 to hear audio information corresponding to thesentiment associated with the selected emoji. Annotated waveform may bedesirable, because it includes an indication of loudness (e.g., lengthof a segment 710 in a vertical direction). Further, clients orapplications on devices 108, 110 can be used to review audio informationon any section annotated waveform 700 by, for example, scrolling acursor over the waveform.

As noted above, database 208 and/or 214 can include sentimentinformation associated with one or more communications. The sentimentinformation can further be associated with one or more users (e.g., auser of device 102), one or more agents, and/or one or more supervisors.

Score calculation module 216 can be used to determine scores, includingaverage, mean, high, low, and the like scores associated with one ormore of a user, agent, or supervisor; such information can be presentedto an agent and/or supervisor using a client or application as describedherein. FIG. 8 illustrates information 802 that includes summaryinformation for multiple agents that can be presented to a supervisorusing the application or client and information 804 for a particularagent that can be displayed to a supervisor using the application orclient. Information 802 and/or 804 can be accessed (e.g., on demand)using an application or agent on device 108 or 110.

Information that can be displayed using a system or method describedhere can vary according to a number of factors. In some cases, each callcan represent an information set for the agent, and the more calls anagent handles, the more information sets become available to asupervisor for that agent. The supervisor can then use the informationsets to, for example, view trends based on an agent's sentiments orcustomers' sentiments when communicating with a particular agent oragents. Such information can be used for performance reviews,situational coaching (e.g., “for every moment you lost your cool, we candiscuss them quickly and review the events leading up to it;” “I seeyou're dealing with a lot of irate customers, which is why you may havegreater unresolved cases,” etc.).

Other trends, such as the agent's best days for performance or besttimes for performance, can also be observed. Overall trends for theentire contact center can also be made from viewing informationassociated with multiple agents. This allows observation of the overalleffectiveness of a contact center, which can be viewed, for example, byspecific hours, days, months or even years. Additional data can be usedto add a precursory state which can affect the agent or customer'soverall sentiment score. The weight of this against their score could befully customizable and scored using score calculation module 216. Forexample, data could be retrieved from social media based off recentstatus updates or posts.

As illustrated, information 802 can be displayed for a selected timeperiod, which can be selected using the application or client. Further,an average score 808 and optionally an emoji 810 corresponding to theaverage score for the selected time period can be displayed for eachagent. The client or application can allow an agent or supervisor toselect an agent 814 from a list of agents 806 to view additionalinformation 804 associated with the selected agent. Additionally oralternatively, the client or application can allow an agent orsupervisor to select an emoji. In this case, information for all agentsor for selected agents that include the selected emoji can be displayedto the agent or supervisor.

Information 804 can include sentiment information for all communicationsfor the selected time frame for a selected agent (e.g., an agent 814selected from information 802). Information 804 can include emojisassociated with sentiment(s) for each call. The application or clientcan allow an agent or supervisor to select an emoji to hear audioinformation and/or view transcribed information associated with theselected sentiment. Selecting the call identification 812 from a list ofcall identifications can cause an annotated waveform, such as anannotated waveform illustrated in FIG. 7 and/or a highlight reel, suchas a highlight reel 600 illustrated in FIG. 6 to be displayed.

Turning now to FIGS. 3-5, exemplary methods in accordance withembodiments of the disclosure are illustrated. The illustrative methodscan be performed using, for example, system 100 and/or sentimentdetermination system 200.

Method 300, illustrated in FIG. 3, includes the step of establishing anelectronic communication (step 302), recording at least a portion of thecommunication (step 304), transcribing at least a portion of a recordingof the communication (step 306), determining one or more sentiments(step 308), determining an emoji associated with the one or moresentiments (step 310), generating a highlight reel (step 312),generating an annotated waveform (step 314), and generating an emojiscore (step 316). Methods in accordance with various examples of thedisclosure need not include all of the illustrated steps and may includevarious combinations of the illustrated steps.

During step 302, a communication is established between a user device(e.g., device 102) and an agent device (e.g., device 108 or 110). Thecommunication can include, for example, an internet protocol (IP)communication.

Once a communication is established, at least a portion of thecommunication is recorded during step 302. The at least a portion of thecommunication can be recorded using, for example, call recorder 202. Therecording can be stored as audio information in a database, such asdatabase 208.

During step 306, at least a portion of the audio information istranscribed to generate transcribed information. The transcribedinformation can be stored in a database, such as database 208. As notedabove, the transcription can be in near real-time (e.g., begin duringthe communication) or can be performed after the communication hasended.

Once the transcription has started or after the transcription iscompleted, one or more sentiments associated with the communication aredetermined during step 308. As noted above, the sentiment(s) can bedetermined using sentiment extraction engine 206.

During step 310, an emoji associated corresponding to the sentimentdetermined from step 308 is determined. The emoji can be determined byquerying a database including sentiment and associated emojiinformation, such as database 208.

During step 312, a highlight reel, such as highlight reel 600, isgenerated. The highlight reel can be interactive, such that a user(e.g., agent or supervisor) can select a portion of the reel and listento corresponding audio information and/or view corresponding transcribedinformation.

Additionally or alternatively, during step 314, an annotated waveform,such as annotated waveform 700, is generated. The annotated waveform canbe generated using overlay production module 212 and can be stored in adatabase, such as database 208 or 214.

During step 316, an average score for a communication, for a customer,for an agent, for a group of agents, or the like can be calculated—e.g.,using score calculation module 216. Individual score informationassociated with a customer, agent, group, or the like can be stored indatabase 214. The scores can be calculated in real time or on demand,such as upon request and selection by a supervisor.

FIG. 4 illustrates method 400 of determining a score or scoresassociated with, for example, one or more agents. Method 400 includesthe step of providing annotated information (step 402) and determining asentiment score (step 404).

During step 402, annotated information from one or more priorcommunications is accessed. A score, such as an average or mean scorecan be calculated based on the annotated information—e.g., using scorecalculation module 216. Agent scores can be displayed, for example, asillustrated in FIG. 8. A prior customer score can be accessed during acall set up and can be used to determine to which agent a call should beconnected. For example, a previously disgruntled or angry customer canbe routed to a senior agent. Method 400 can, for example, allow asupervisor to review summaries of communications for particular agentsfor, for example, selectable time periods, selectable topics, or thelike.

FIG. 5 illustrates another method 500 in accordance with embodiments ofthe disclosure. Method 500 includes the steps of providing annotatedinformation (step 502), determining a prior sentiment score (step 504),determining a current sentiment score (step 506), and determining acomparison or weighted score (step 508).

During step 502, annotated information from a previous communicationthat is stored in a database can be retrieved. The sentiment informationcan also be retrieved or can be determined during step 504. During step506, the sentiment of the current communication is determined—e.g.,using techniques described herein. A weighted average of the current andone or more prior communications are determined during step 508—e.g.,using score calculation module 216. The weighted score can be reviewedby a supervisor to see if an improvement or decline in sentimentoccurred. A number of prior communications can be selected by asupervisor or may be filtered to relate to a specific topic.

The present invention has been described above with reference to anumber of exemplary embodiments and examples. It should be appreciatedthat the particular embodiments shown and described herein areillustrative of the invention and its best mode and are not intended tolimit in any way the scope of the invention as set forth in the claims.The features of the various embodiments may stand alone or be combinedin any combination. Further, unless otherwise noted, various illustratedsteps of a method can be performed sequentially or at the same time, andnot necessarily be performed in the order illustrated. It will berecognized that changes and modifications may be made to the exemplaryembodiments without departing from the scope of the present invention.For example, although much of the disclosure relates to a call center,the methods and systems can be used for other applications, such asconversion of hardware or software states into emoji-based functions,each with dynamically generated interface elements. For example, a logfile that highlights failure conditions or events could be parsed thensummarized using a highlight reel. Performance indicators for virtuallyany device could be more easily summarized using universally acceptedemojis as a summary-oriented lexicon to access log events. These andother changes or modifications are intended to be included within thescope of the present invention, as expressed in the following claims.

1. An electronic communication system comprising: a communicationserver; a sentiment determination system coupled to the communicationserver; and one or more devices coupled to the communication server,wherein the sentiment determination system determines a sentiment duringan electronic communication and generates information comprising anemoji corresponding to the sentiment.
 2. The electronic communicationsystem of claim 1, further comprising a call recorder to record audioinformation during an electronic communication between the one or moredevices and the communication server.
 3. The electronic communicationsystem of claim 2, further comprising a transcription module totranscribe the audio information.
 4. The electronic communication systemof claim 1, wherein the sentiment determination system comprises asentiment extraction engine.
 5. The electronic communication system ofclaim 1, further comprising a database comprising emoji information andcorresponding sentiment information.
 6. The electronic communicationsystem of claim 1, further comprising a highlight reel production moduleto generate information annotated with one or more emojis indicative ofsentiment.
 7. The electronic communication system of claim 1, furthercomprising an overlay production module to generate an acoustic waveformannotated with one or more emojis indicative of sentiment.
 8. Theelectronic communication system of claim 1, further comprising a scorecalculation module.
 9. The electronic communication system of claim 8,wherein the score calculation module determines a sentiment scoreassociated with a caller.
 10. The electronic communication system ofclaim 8, wherein the score calculation module determines a sentimentscore associated with an agent.
 11. The electronic communication systemof claim 1, wherein the system generates a sentiment graph linked to oneor more sections of the audio information.
 12. An electroniccommunication method comprising the steps of: establishing an electroniccommunication; transcribing at least a portion of a recording of thecommunication to generate transcribed information; determining one ormore sentiments associated with the at least a portion of thetranscribed information; and generating annotated information comprisingone or more emojis associated with each of the one or more sentiments.13. The electronic communication method of claim 12, further comprisingthe step of generating an audio recording of at least a portion of thecommunication.
 14. The electronic communication method of claim 12,further comprising the step of generating a highlight reel.
 15. Theelectronic communication method of claim 12, further comprising the stepof generating an acoustic waveform annotated with one or more emojisindicative of sentiment.
 16. The electronic communication method ofclaim 12, further comprising the step of generating an emoji scoreassociated with a customer.
 17. The electronic communication method ofclaim 12, further comprising the step of generating an emoji scoreassociated with an agent.
 18. The electronic communication method ofclaim 12, further comprising the step of generating a summary of emojiscores associated with the agent.
 19. The electronic communicationmethod of claim 12, further comprising the steps of: generating an emojiscore for the electronic communication; and comparing the emoji scorefor the electronic communication to a prior communication emoji score.20. An electronic communication system comprising: a communicationserver; a sentiment determination system coupled to the communicationserver; and one or more devices coupled to the communication server,wherein the sentiment determination system comprises: a sentimentextraction engine; and a database comprising emoji information andcorresponding sentiment information.